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Abstract 

The ((i, a, /5, 7)-brancliing particle system consists of particles moving in according to 
a symmetric a-stable Levy process (0 < a < 2), splitting with a critical (1 + /?)-branching 
law (0 < /? < 1), and starting from an inhomogeneous Poisson random measure with intensity 
measure fi^{dx) = dx/{l + |x|'^),7 > 0. By means of time rescaling T and Poisson intensity 
measure H^fi^, occupation time fluctuation limits for the system as T — > cxd have been obtained 
in two special cases: Lebesgue measure (7 = 0, the homogeneous case), and finite measures (7 > 
d). In some cases Hj- = 1 and in others Ht — > cxd as T — »• cxd (high density systems). The limit 
processes are quite different for Lebesgue and for finite measures. Therefore the question arises 
of what kinds of limits can be obtained for Poisson intensity measures that are intermediate 
between Lebesgue measure and finite measures. In this paper the measures /i7,7 G {0,d\, 
are used for investigating this question. Occupation time fluctuation limits are obtained which 
interpolate in some way between the two previous extreme cases. The limit processes depend on 
different arrangements of the parameters d, a, /3, 7. There are two thresholds for the dimension 
d. The first one, d = a//? + 7, determines the need for high density or not in order to obtain 
non-trivial limits, and its relation with a.s. local extinction of the system is discussed. The 
second one, d = [a(2+/5)— 7Vq;)]//5 (if 7 < d), interpolates between the two extreme cases, and 
it is a critical dimension which separates different qualitative behaviors of the limit processes, in 
particular long-range dependence in "low" dimensions, and independent increments in "high" 
dimensions. In low dimensions the temporal part of the limit process is a new self-similar stable 
process which has two different long-range dependence regimes depending on relationships 
among the parameters. Related results for the corresponding (rf, a, /?, 7)-superprocess are also 
given. 
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1 Introduction 



Occupation time fluctuation limits have been proved for the so-called (rf, a, /3)-branching 
particle systems in M!^ with initial Poisson states in two special cases, namely, if the Poisson in- 
tensity measure is either Lebesgue measure, denoted by A, or a flnite measure |BGT1] . |BGT2j . 
|BGT3] . |BGT4] . |BGT6] . Those cases are quite special, as explained below, and the limit pro- 
cesses are very different. Therefore the question arises of what happens with Poisson intensity 
measures that are intermediate between Lebesgue measure and flnite measures. That is the 
main motivation for the present paper, and our aim is to obtain limit processes that interpolate 
in some way between those of the two special cases. One of our objectives is to flnd out when 
the limits have long-range dependence behavior and to describe it. Another motivation is to 
derive analogous results for the corresponding sup erpro cesses. 

In a (c?, a, /3) -branching particle system the particles move independently in according 
to a standard spherically symmetric a-stable Levy process, < a < 2, the particle lifetime is 
exponentially distributed with parameter V, and the branching law is critical with generating 
function 

s + ^{i-sy^^ o<s<i, (1.1) 

where < (3 < 1 (called (1 + /3)-branching law), which is binary branching for j3 = 1. The 
parameter V is not particularly relevant, but it is convenient to use it. The empirical measure 
process N = (A^t)t>o is deflned by 

Nt{A) = number of particles in the Borel set A cW^ a.t time t. (1-2) 

A common assumption for the initial distribution A'^o is to take a Poisson random measure with 
locally flnite intensity measure /z. The corresponding (c?, a, /5)-superprocess Y = (Yt)t>o is a 
measure- valued process, which is a high-density/short-life/small-particle limit of the particle 
system, with Yq = /i. See jD], [E], |P2j for background on those particle systems and super- 
processes. In this paper we investigate (the limiting behavior of) the corresponding occupation 
time processes, i.e., 

/ Nsds, t > 0, and / Y^ds, t > 0. 
Jo Jo 

We recall that the distributions of these processes are characterized by their Laplace functionals 
as follows [GLM] . jDP2] : 

^expj-^ (Ar„(^)rfs| =exp{-(/i,w^(t))}, G 5(M''), (1.3) 

where v^{x,t) is the unique (mild) solution of the non-linear equation 



d . V 



^aV^ - T^vl^^ + ^{l - V^), (1.4) 



and 

ft 



^exp<^- / {Y,,ip)ds} =exp{-{ij,u^{t))}, ^ G 5(M^), (1.5) 
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where u^{x,t) is the unique (mild) solution of the non-linear equation 





(1.6) 



u^{x,0) = 0, 



and Aa is the infinitesimal generator of the a-stable process. (See the end of the Introduction 
for the standard notations ( , ),S{M.'^).) 

For fi = X, No is homogeneous Poisson. This case is special (and technically simpler) because 
A is invariant for the a-stable process (which implies in particular that ENt = A for all t) , and 
there is the following persistence/extinction dichotomy |GW] . which heuristically explains the 
need for high density in some cases in order to obtain non-trivial occupation time fiuctuation 
limits, and anticipates the situation we will encounter in this paper: 

(i) Persistence: If ci > a/ (3, then Nt converges in law to an equilibrium state A^oo as t ^ oo, 
such that ENoo = A. 

(ii) Extinction: If c? < a/ (3, then Nt becomes locally extinct in probability as t oo, i.e., for 
any bounded Borel set A, Nt{A) — > in probability. 

An analogous persistence/extinction dichotomy holds for the corresponding superprocess (with 
Yo = A) |DPlj . For a = 2 (Brownian motion) and d < 2/ (3 a stronger extinction holds: the 
superprocess becomes locally extinct in finite time a.s. [12], and we shall see that so does the 
particle system. 

The case of /z finite is special because the particle system goes to extinction globally in finite 
time a.s. for every dimension d, and so does the superprocess |P2j . 

The time-rescaled occupation time fiuctuation process Xt = (-^r(^))t>o of the particle 
system is defined by 



where Ng is given by (11. 2p and is a norming. The problem is to find F^ such that 
converges in distribution (in some way) as T — oo, and to identify the limit process and study 
its properties. This was done for /i = A in the persistence case, d > a/ (3, |BGT3j . |BGT4j . 
For the extinction case, d < a/f3, in [BGT6j we introduced high density, meaning that the 
initial Poisson intensity measure was taken of the form H^X, with Hx ^ oo as T ^ oo, so as 
to counteract the local extinction, and we obtained similarly high-density limits with fi finite. 
For n = X and d > a/(3 the same results hold with or without high density (with different 
normings) |BGT6] . The limit processes are different for fi = X and fi finite (some differences 
are mentioned below), and the results for any finite measure are essentially the same. 

In order to study asymptotics of Xt as T — oo with measures /i that are intermediate 
between the two previous cases, we consider Poisson intensity measures of the form 



We call the model so defined a (d, a, /?, 7)- branching particle system, and a. {d,a, P, 7)-superpro- 
cess the corresponding measure-valued process. To obtain non-trivial limits we multiply fi^ by 
Ht, which is suitably chosen in each case. Since fio = X and fi^ is finite for 7 > d, by varying 
7 in the interval (0, d] we obtain limits that are between those of the two previous cases, which 




(1.7) 




1 



dx, 7 > 0. 



(1.8) 



1 + b|T 
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are extreme in this sense, in a way that interpolates between them. The substantial role of 7 
was already noted in the simpler model of particle systems without branching |BGT5j . 

The above mentioned behaviors of Nt in the cases 7 = and 'j > d raise the following 
questions on what happens for 7 G {0,d], and on its effect on asymptotics of Xt: When does 
Nt suffer a.s. local extinction in the sense that for each bounded Borel set A there is a finite 
random time such that Nt{A) = for all t > a.s.? In this case the total occupation time 

Nt{A)dt is finite a.s., and therefore high density is needed in order to obtain non-trivial 
limits for Xt- For 7 > 0, Nt{A) converges to in probability as t —>■ 00 for any bounded Borel 
set A and every dimension d, so local extinction in probability occurs, but the total occupation 
time may or may not be finite. It turns out that the threshold between the need for high density 
or not is given by ci = + 7, and then a natural question is whether d = a/P + 'y is also the 
border to a.s. local extinction of the particle system. We will come back to this question. 

The limits for Xt in |BGT6] are of three different kinds for both fi = \ and n finite. In the 
first case there is a critical dimension, dc = a{l + [3)/ (3. For the "low" dimensions, d < dc, the 
limit has a simple spatial structure (the measure A) and a complex temporal structure (with 
long-range dependence). For the "high" dimensions, d > dc, the limit has a complex spatial 
structure (distribution-valued) and a simple temporal structure (with stationary independent 
increments). For the "critical" dimension, d = dc, the spatial and the temporal structures are 
both simple, but the order of the fluctuations (Ft) is larger, as is typical in phase transitions. 
The limit processes are always continuous for d < dc, and for d > dc they are continuous if and 
only if /3 = 1 (when the limits are Gaussian). For /i finite, an analogous trichotomy of results 
holds, with a new critical dimension, dc = a(2 + /3)/(l + f3), another difference being that the 
limits for the critical and high dimensions are constant in time for t > 0. 

In this paper we show analogous limits of Xt for {d, a, /3, 7)-branching particle system; the 
critical dimension changes between the ones above, a(l -|-/9)//9 for 7 = 0, and a{2 + /?)/(! -|-/5) 
for J > d, and they are linked with a unified formula, 

4(7) = a— —, (1.9) 

which interpolates between the two cases (see Remark 2.2(a) for a precise statement). There are 
several limit processes depending on different arrangements of d, a, (5, 7. Some are analogous 
to those for /i = A, and some are similar to those for /i finite (or even essentially the same). 
For 7 < d there are six different cases that include the three ones recalled above for 7 = 0. 
For 7 > d there are the three cases obtained in |BGT6j (generally for finite /x). In the case 
7 < (i and d < dd'j), the temporal structure of the limit is a new real, stable, self-similar, 
continuous, long-range dependence process ^, defined in (2.1) below, which has two different 
long-range dependence regimes if a < 2 (Theorem 1(a) and Proposition 2.3). This strange type 
of long-range dependence behavior already appears in the homogeneous case, 7 = |BGT3j . 
|BGT6] . An analogous phenomenon occurs with < 7 < d, the border between the two long- 
range dependence regimes changes continuously with 7, and it disappears in the limit ■y d 
(see formula (2.13)). For j > d there is only one long-range dependence regime, not depending 

on -f,f3 

In [BGTlj . |BGT2] . |BGT3] . |BGT4] . |BGT5] . |BGT6] we have given the convergence results 
for Xt in a strong form (functional convergence when it holds), but in the present article our 
main objective is identifying the limits, so we have not attempted to prove the strongest form 
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of convergence in each case, nevertheless we expect that convergence in law in a space of 
continuous functions holds in all cases where the limit is continuous. We prove functional 
convergence only in the case of the above mentioned long-range dependence process ^ because 
of its special properties. A technical difficulty for the tightness proof is the lack of moments if 



The time-rescaled occupation time fluctuation process for the {d, a, P, 7)-superprocess Y is 
defined analogously as fll.7p . 



Jo 

and the limits are obtained from (the proofs of) the results for the {d, a, (3, 7)-branching particle 
systems, as a consequence of the fact that the log-Laplace equation of the occupation time of 
the superprocess is simpler than that of the particle system (see (11. 3p . (II. 4p . and (II. 5p . (II. 6p .) 

Our results on the fluctuation limits of superprocesses generalize those of Iscoe [11], who 
considered the homogeneous case (7 = 0) only. 

Let us come back to the question of high density and local extinction for the (ci, 7)- 
branching particle system. From Theorems 2.1, 2.5 and 2.6 it follows immediately (see Corol- 
lary 2.10) that in all cases where high density is not necessary (i.e., we may take Ht = 1) 
there is no a.s. local extinction (in spite of the fact that local extinction in probability occurs 
if 7 > 0). For instance, condition (12. 6p in Theorem 2.1(a) holds automatically if d > a//3 + ^ 
(with 7 < (i), hence high density is not necessary for a non-trivial limit of Xt in this case. On 
the other hand, high density is indispensable to obtain a non-trivial limit if either d < a/jS + 'j 
or a < 'J < d. In the latter case the total occupation time of any bounded Borel set by the 
process N is finite a.s. (it has finite mean). We prove a.s. local extinction for a = 2 and 
d < 2/(3 + •y, and we conjecture that a.s. local extinction holds generally for d < + 7 also 
if a < 2. This conjecture is supported by the fact that for c? = -|- 7, 7 < a, there is an 
ergodic result (Proposition 2.9). For a = 2 and d < 2//5 + 7 it follows from |l2j (Theorem 8/3) 
that the (d, 2, /3, 7)-superprocess suffers a.s. local extinction. The method of [Z] can also be 
used to prove this (private communication). The proof of a.s. local extinction of the particle 
system consists in showing that a.s. local extinction of the (c?, 2, /3, 7)-superprocess implies a.s. 
local extinction of the (c?, 2, /3, 7) -branching particle system (Theorem 2.8). This implication is 
not as simple as it might seem because the well-known Cox relationship between the particle 
system and the superprocess (i.e., for each t, Nf is a doubly stochastic Poisson random measure 
with random intensity measure given by Y^) is not enough to relate the long time behaviors of 
the two processes. But the argument does not work for a < 2. In this case the superprocess Y 
has the instantaneous propagation of support property, i.e., with probability 1 for each t > if 
the closed support of Yt is not empty, then it is all of W^. This follows from the result proved 
in |Plj for finite initial measure and finite variance branching (/5 = 1 in our model), which is 
extended in [LZj for more general superprocesses and branching mechanisms (including (3 < 1 
in our case). It follows that for the (rf, a, /3, 7)-superprocess with a < 2, a.s. local extinction 
and global extinction are equivalent, and it is known that if the initial measure has infinite total 
mass, the probability of global extinction in finite time is 0. Nevertheless, the total occupation 
time of a bounded set for the superprocess with a < 2 may or may not be finite, and this is 
what is directly relevant for us (see the proof of Theorem 2.8). Iscoe [IT] showed that for initial 
Lebesgue measure and a = 2 the total occupation time of a bounded set is finite if and only if 



(3<1. 




(1.10) 
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(i < 2//?, and he conjectured that an analogous result holds for a < 2 and d < a/p. So far as 
we know, this conjecture has not been proved. 

Summarizing, if 7 < (i and 7 < a, there are two thresholds for the asymptotics of X^, 
namely, + 7, and dcij) given by (11.91) . The first one, which is smaller than the second 
one, appears to be the border to a.s. local extinction (we know that it is for a = 2), and 
it determines the need for high density. The second one is the critical dimension between 
changes of behavior of the limit processes, in particular the change from long-range dependence 
to independent increments, and from continuity to discontinuity ii j3 < 1. An interpretation 
of dci'j) in terms of the model seems rather mysterious, even in the case 7 = (see [BGT4j . 
Section 4, for several questions on the meaning of results). 

The general methods of proof developed in |BGT3] . |BGT4] . [BGT6j . and the special cases 
for (3 = 1, where the limits are Gaussian |BGT1] . |BGT2] . can be used for the proofs involving 
fi^. However, a considerable amount of technical work is unavoidable in order to deal with 
7 > 0. Moreover, each case requires different calculations. We will abbreviate the proofs as 
much as possible. 

Related work appears in |BZ] . [Ml] , |M2j for the special case 7 = 0, /5 = 1. |BZ] studies 
occupation time fluctuations of a single point for a system of binary branching random walks 
on the lattice with state dependent branching rate. [Mlj . [M2j consider general critical finite 
variance branching laws. |BZ] . [Ml] , |M2j and |M3j also study the systems in equilibrium. 
We have already mentioned the paper [IT] on occupation time fiuctuation limits of {d,a,j3)- 
superprocesses. Some other papers regarding extinction, ergodicity and occupation times of 
branching particle systems and superprocesses are |BZ] . |CG] . |DGW] . |DR] . |DFj . [EKj . |FG] . 
jFVW] . [H], [IL], [K], [LR], [MRl, [MH], |Shj, [E], [VWj, [Zh] (and references therein). 

We have given special attention to the long-range dependence stable process ^ and its prop- 
erties because long-range dependence is currently a subject of much interest (see e.g. [DOTj . 
|H1] . |H2] . [S], [T] for discussions and literature), hence it is worthwhile to study different types 
of stochastic models where it appears. Other types of long-range dependence processes have 
been found recently (e.g. [CS], [GNRj . [HJ] . [HVj . [KT], [MY], [LT], [FTL] ). in particular in 
models involving heavy-tailed distributions. 

The following notation is used in the paper. 

iS(M'^): space of rapidly decreasing functions on W^. 

iS'(M'^): space of tempered distributions (topological dual of iS(M'^)). 

( , ): duality on S'{R'^) x S{R'^), or on >S'(ffi'^+^) x 5(M'^+^), in particular, integral of a 
function with respect to a tempered measure. 

=^c'- weak convergence on the space of continuous functions C([0, r], 5'(]R'^)) for each r > 0. 

=^f: weak convergence of finite-dimensional distributions of iS'(M'^)-valued processes. 

=^i: integral convergence of 5'(M'')-valued processes, i.e., Xt X if, for any r > 0, 
the iS'(M'^'''^)-valued random variables Xt converge in law to X as T — 00, where X (and, 
analogously X^) is defined as a space-time random field by 

{X,^)= r{X{t),^;t))dt, $G5(M^+1). 
^0 

=^f/- =^f and together. 

Recall that in general =^ / and do not imply each other, and either one of them, together 
with tightness of {(XT,<^)}r>i in C([0,r],M), t > 0, cp e 5(M'^), implies [BGR] . 
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The transition probability density, the semigroup, and the potential operator of the standard 
symmetric a-stable Levy process on are denoted respectively by Pt{x), % (i.e., %Lp = pt* Lp) 
and (for d > a) 

Gcpix) = r%v{x)dt = / —M^dy, (1.12) 

where 

r(^) 

^"''^ ^ 2°7r^^72r(f) ' (^-^3) 

Generic constants are written, C, Ci, with possible dependencies in parenthesis. 
Section 2 contains the results, and Section 3 the proofs. 



2. Results 

Given /3 G (0,1], let M be an independently scattered (1 + /3)-stable measure on W^^^ 
with control measure A^+i (Lebesgue measure) and skewness intensity 1, i.e., for each Borel 
set A C R'^^^ such that < Xd+iiA) < oo,M{A) is a (1 + /?) -stable random variable with 
characteristic function 



exp|-Arf+i(A)|2;|i+^ (l-t{sgnz) tan|(l + /?) 



z e 



the values of M are independent on disjoint sets, and M is cr-additive a.s. (see [STj . Definition 
3.3.1). 

For a G (0, 2], 7 > 0, we define the process 

it= I \Moa{^)[ Pr{x-y)\y\-^dy] pu-r{x)du ] M{drdx), t > 0, (2.1) 



which is well defined provided that 

Pr{x — y)\y\ '^dy ij pu-r{x)du j drdx < 00, (2.2) 

(see jSTj ). For 7 = 0, ^ is the same as the process ^ defined by (2.1) in jBGTGj . We also recall 
the following process defined by (2.2) in [BGTGj . 

Ct= [ (l[o,t]{r)py'-'''''\x) [ Pu-r{x)du]M{drdx), t>0 



[o,t]ir)py^'-^^\x) Pu-r{x)du]M{drdx), t > 0, (2.3) 

which is well defined if < a{2 + (3) /{I + (3). 

We consider the {d, a, (3, 7)-branching particle system described in the Introduction with 
defined by (11.71) . Recall that the initial Poisson intensity measure is Hx^i-y We formulate the 
results for low, critical and high dimensions separately, since, as mentioned in the Introduction, 
the qualitative behaviors of the limit processes are different in each one of these cases. In the 
theorems below K & positive number depending on d^ a, /3, 7, V, which may vary from case 
to case and it is possible to compute it explicitly. 

The results for the low dimensions are contained in the following theorem. 
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Theorem 2.1 (a) Assume 'j < d and 



d<a- -. (2.4) 

Then the process ^ given by \2. 1\) is well defined, and for 

= H^T^+P-idP+y)/a ^2.5) 



rpl-{d-j)l3/a 

lim = 0, (2.6) 



and put 

Then for 
with 



with Ht > 1 and 



we have Xt =^c KXC,. 
(h) Let 7 > rf, 

d<ai±|. (2.7) 

4+^^ = HTT^+^-^^+l'^^'''k{T) (2.9) 
T 

lim — ^ = 0, (2.10) 

T^^H^k{TY ^ ' 

we have Xt =^f,i KX(, where ( is defined by Ii2.3\) . 

Remark 2.2 (a) For 7 = 0, Theorem 2.1(a) is the same as Theorem 2.2(a) in |BGT6] . For 
7 < a and 7 < (i, the bound on the dimension remains the same, equal to a(l + (see 
fl2.4p ). and for a < 7 < d, it changes continuously, tending to the threshold (12. 7p a.s •y d. 

(b) For d satisfying (12. 4p and additionally d > a/P + 'y, condition (12.61) holds with Ht = 1, so 
in this case high density is not needed, and the limit of Xt is the same as for the high-density 
model. 

(c) The case 7 > c? is included for completeness only, since it is contained in Theorem 2.7 of 
[BGT6j , where a general finite intensity measure was considered. The same remark applies also 
to the theorems for critical and high dimensions (Theorems 2.5 and 2.6 below). 

(d) Note that the limit process is the same (up to constant) for the infinite intensity measure 
HTdx/{l + \x\'^) (7 = d) as for finite measures. 

(e) In Theorem 2.1(b) we consider convergence =^f,i only. We are sure that functional con- 
vergence holds (in fact, for the case 7 > c/ this was proved in |BGT6] ). but, as stated in the 
Introduction, we are mainly interested in the identification of limits and we do not attempt to 
give convergence results in the strongest forms. The same applies to the theorems that follow. 

In the next proposition we gather some basic properties of the process ^ defined by (12. ip . 
in particular its long-range dependence property. (The process C, was discussed in |BGT6] ). In 



8 



|BGT3j we introduced a way of measuring long-range dependence in terms of the dependence 
exponent, defined by 



K= inf inf suplO > : Dt{zi, Z2;u,v, s,t) = o{T ^) as T — > cx)}, (2-11) 

zi,Z2eM 0<Ji<'!;<s<t 

where 

DTizi,Z2;u,v,s,t) 

= I logii^e**-^^*-^"~^"-'^^^*-^^+*~^^+'''' — logE'e*^^*-^"^^"-' — logi^^e*^^*-^^+*~^^+''''|. (2.12) 
(see also |RZj ). 

Proposition 2.3 Assume 'j < d and ^2.4^ . Then 

(a) ^ is {1 + P)-stable, totally skewed to the right if (3 <1. 

(h) ^ is self-similar with index (2 + /3 — {d[3 — 7)/a)/(l + 

(c) ^ has continuous paths. 

(d) ^ has the long-range dependence property with dependence exponent 

{ - if either a = 2, or a <2 and B > 

+ ^/ «<2 and /5<^. ^'-''^ 

Remark 2.4 (a) Here, as in the case 7 = (Theorem 2.7 of [BGT3j ). the intriguing phe- 
nomenon of two long-range dependence regimes occurs for a < 2. It seems also interesting to 
note that putting formally 7 > ci in (12.131) we obtain k = d/a (with no change of regime), 
which is indeed the dependence exponent of the process ( (Proposition 2.9 of [BGT6j ). On the 
other hand, the process ( itself is not obtained from ^ by putting •j > d. 

(b) If 7 = and (3=1, then ^ is the sub- fractional Brownian motion (multiplied by a 
constant) considered in |BGTj . |BGTlj . 

We now turn to the critical dimensions, i.e., the cases where the inequalities in (12. 4p and 
(12. 7p are replaced by equalities. It turns out that in spite of different conditions on the normings, 
the limits have always the same form as for finite intensity measure, with the only exception of 
the case given in Theorem 12.5( a) below. 



Theorem 2.5 (a) Assume 7 < ci, 7 < a, 



rf = «^^ (2.14) 



and 



F^+f^ = HtT^-^/'' log T, (2.15) 

with Ht > 1. Then Xt =^f,i KXrj, where rj is a {1 + f3)-stahle process with independent, 
non- stationary increments (for 7 > whose laws are determined by 

= exp|-(ti-^/" - s^-^/")|z|^+^ (^1 -i(sgn;z)tan|(l + /5))| , z eR, t > s > 0, 

Vo = 0. 
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(b) In all the remaining critical cases, i.e., 
(i) '-y = a,'-y < d with d satisfying ^2.14^ , 



4+'^ = i^T(log^)^ (2.16) 



and 



(a) a < 'J < d with 



lira^^^^ =0, (2.17) 



(2.18) 

F^-^f = HrlogT, (2.19) 



and 
(Hi) 

with Ft satisfying 112. 1 b]) and 



2^(l+/3)(7/a-l) 



lini. = 0, (2.20) 



7 = d = a^, (2.21) 
T 

lim — ^ = 0, (2.22) 

^--i/^ (log T) 1+2/3 

(iv) 7 > d = a(2 + /3)/(l + (3), Fy^'^ = i^T logT and limr-.oo Ti/^^ = 0, 

we have Xt =^f,i KX'd, where is a real process such that t^o = and for t > 0,^t = = 
(1 + P)-stable random variable totally skewed to the right, i.e., 

Ee''^' =exp|-|2|i+'^ (^l-i(sgnz)tan|(l + /5))}. 

It remains to consider the high dimensions. 
Theorem 2.6 (a) Assume 'j < d,'j < a, 

d>a^-^, (2.23) 

and 

= HtT^-^/'^, (2.24) 

with Ht > 1. Then =^f,i X, where X is an S'iR'^) -valued (1 + (3)-stable process with 
independent, non- stationary increments (for 7 > j determined by 

= exp| -ir(t^-^/"-s^-^/°) / fl^|G'(^(a;)|^+''(l -i(sgnG(^(x))tan-(l + ^)) 
I J^d\ 2 

+ 2cp^{x)G^{x)jdx^, i^eS{W^), t > s > 0, (2.25) 
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Xq = 0, where 

and G is defined by (OH^^- 

(b) Assume 7 < 7 = a, and d satisfying Ii2.23\) with 

F^+z^ = ifrlogT, (2.27) 

and Ht > 1- Then Xt =^f,i X, where X is an S'{W^)-valued process such that Xq = Q, and for 
t > 0, X( = Xi = (1 + (3)-stable random variable determined by 

^gi(Xi,^> _ exp\-K [ fv|G^(x)|i+^(l -i(sgnG(^(x))tan-(l + /5)) 
I Jigd V 2 

+ 2cf3ip{x)Gip{x)^dxy ¥?g5(R"'), (2.28) 



with C/3 given by Ii2.26\) . 

(c) Assume 7 > a, 



2 + j3 7Ad 
d>a^ - (2.29) 



= Ht, (2.30) 



and 



lim Ti/^^ = 0. (2.31) 

T — ^00 

Then Xt =^f,i X, where X is an S'{W^)-valued process such that Xq = 0, and for t > 0, Xt = 
Xi = (1 + P)-stable random variable determined by 

Ee'^^"^^ = exp\-K I (v\G^{x)\^+'^{l - i{sgnG^{x))ia.n-{l + (3)) 

+ 2c^(^(x)G'(^(x))G/i^(dx)}, (^e5(M'^), (2.32) 

with C/3 given by /12.26\) . 

Remark 2.7 (a) As in all the cases studied previously |BGT3] . |BGT4] . |BGT6] . we observe 
the same phenomenon that in low dimensions the limit processes are continuous with a simple 
spatial structure and a complicated temporal structure (with long-range dependence), while in 
high dimensions they are truly iS'(R'^)-valued with independent increments, and not necessarily 
continuous. 

(b) For low dimensions the forms of the limits depend on the relation between d and 7 only, 
whereas for critical and high dimensions only the relationship between a and 7 is relevant. 
More precisely, in critical dimensions we have different forms of the limits for 7 < a and 7 > a, 
and in high dimensions the forms are different for 7 < a, 7 = a and 7 > a. In the case 7 > a 
even the normings are the same, depending only on f3. 

(c) For [3 = 1 the limits are centered Gaussian. In high dimensions there is no continuous 
transition between the cases (3 < 1 and /3 = 1; in the latter case an additional term appears. 
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The coefficient C/3 defined in fl2.26l) was introduced in order to present the results in unified 
forms. 

(d) We have assumed that the initial intensity measure is determined by /i^ of the form fll.Sp . 
It will be clear that the same results are obtained for the measure n^{dx) = \x\~'^dx if > 7. 
Analogously as in the non-branching case |BGT5] . other generalizations are also possible. 

Let us look further into the need for high density (i.e., to assume Ht 00) and the question 
of a.s. local extinction. For 7 > d the Poisson intensity measure is finite, so there is only a 
finite number of particles at time t = and the system becomes globally extinct in finite time 
a.s. due to the criticality of the branching. Also, for 7 A li > a it is not difficult to see that 
the total occupation time Ngds is finite a.s. on bounded sets (see |BGT5] . Proposition 2.1, 
because ENg is the same for the systems with and without branching), so high density is also 
necessary. We have a more delicate situation in the remaining cases where the threshold is 
d = a/ /3 + 'J. Concerning extinction, the situation is completely clear for a = 2. In Theorem 
2.8 below we state that for a = 2 and d < 2//5 + 7 there is a.s. local extinction, hence the 
total occupation time of any bounded set is finite a.s. We conjecture that the same is true for 
(i<a//9 + 7ifa;<2, but we have not been able to prove it. 

Theorem 2.8 Assume a = 2. //c? < 2//5 + 7, then for each bounded Borel set A, 
P[there exists Tj^ < 00 such that Nt{A) = for all t > r^] = 1. 

The proof of this theorem relies on Iscoe's a.s. local extinction result for the superprocess 
[I2] . by showing that in general (i.e., for < a < 2) a.s. local finiteness of the total occupation 
time of the (rf, a, /3, 7) -superprocess implies a.s. local extinction of the (rf, a, /3, 7)-branching 
particle system. On the other hand, as explained in the Introduction, for a < 2 the a.s. local 
extinction for the superprocess cannot occur, and we do not know how to prove directly the 
a.s. local finiteness of its total occupation time. 

The next ergodic-type result, which is a direct generalization of |Ta] . shows that a//? + 7 is 
indeed a natural threshold. 



Proposition 2.9 Assume 

'y <' n d = 



7 < a, d=-+-i, Ft = T^-t/", (2.33) 



and denote 



1 



Zrit) = — / Nsds, t > 0. (2.34) 
Ft Jo 

Then =^c where ^ is a real non-negative process with Laplace transform 

Ee^-Oi^t, = exp (- / v{x,T)\x\-^dx\ , (2.35) 

for any t > 0, where Oi, ... ,9n > 0,0 < ti <■■■< t^ < r, and v{x,t) is the unique non- 
negative solution of the equation 



t 1/ rt 



vix,t)= I p^_^{x)^{t - s)ds - I Tt.sv'^^{;s)ix)ds, (2.36) 
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with 

n 

^(s) = ^41[o,t,](.). (2.37) 
fc=i 

To complete the discussion on a.s. local extinction we formulate a corollary which follows 
immediately from our results but which, nevertheless, seems worth stating explicitly. 

Corollary 2.10 //7 < a and d > a/ P + ■y, then the {d, a, Pj'j) -branching particle system does 
not have the a.s. local extinction property. 

Indeed, for d > a/l3 + j, from Theorems 2.1, 2.5 and 2.6 it follows that one may take 
Ht = 1. By Proposition 2.1 of |BGT5] . E {Ns,ip)ds (for f^af{x)dx ^ 0) is of larger order 
than as T oo hence, by ( 11.71) . for any bounded Borel set A, Jq°° Ns{A)ds = oo a.s., which 
excludes a.s. local extinction. For d = a/ (3 + 'j the result follows immediately from Proposition 
2.9. 

We end with the results for the superprocess. 

Theorem 2.11 Let Y be the {d,a, (3,'y)-superprocess and Xf its occupation time fluctuation 
process defined by (1.10). Then the limit results for Xt as T ^ oo are the same as those in 
Theorems 2.1, 2.5 and 2.6, with the same normings, and Cf3 = in all cases in Theorem 2.6. 



3. Proofs 

3.1 Scheme of proofs 

The proofs of Theorems 2.1, 2.5 and 2.6 follow the general scheme presented in |BGT6] . 
For completeness we recall the main steps. 

As explained in |BGT3t IBGT4t IBGT6] , in order to prove convergence ^ » it suffices to show 

lim Ee-<^^'*> = Ee-<^'*> (3.1) 

T — ^oo 

for each $ G iS(M'^^^), $ > 0, where X is the corresponding limit process and Xt, X are defined 
by (11.111) . To prove convergence according to the space-time approach [BGR] it is enough 
to show additionally that the family {{Xt,^)}t>i is tight in C((0, 

Without loss of generality we may fix r = 1 (see (11.111) ). To simplify slightly the calculations 
we consider $ of the form 

$(a;, t) =(p® ^p{x, t) = ip{x)^{t), if G SiW^),^ G <S(M), y?, > 0. 

Denote ^ 

= X{t) = ^ i^is)ds, xAt) = X (I) • (3.2) 

We define 

VT{x,t) = 1- Eexpi^- j^{N^,ipT)XT{T -t + r)drY < t < T, (3.3) 
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where is the empirical process of the branching system started from a single particle at x. 
The following equation for vt was derived in [BGT3j (formula fl3.8l) . see also |BGTlj ) by means 
of the Feynman-Kac formula: 



VT{x,t) 



Tt-u 



ifTxAT - u){l - vt{-,u)) 



V 



1+/3/ 



{x)du, 0<t<T. (3.4) 



This equation (with T = 1) is the space-time version of equation (1.4). It is the log-Laplace 
equation for L (as in (1.11)), where L is the occupation time Lt = Nsds. Formulas (13.31) and 
(EH) imply 

<vt <l, VT{x,t) < / Tt-uVT{x)xT{T - u)du. 

Jo 

For brevity we denote 



VT{dx) = HTfi'yidx) 
By the Poisson property and ( 13. 4p we have 



1 + \xp 



-dx. 



(3.5) 
(3.6) 



exp 
exp 



vt{x, T)h'T{dx) + 



Tu(pT{x)xT{u)duh'T{dx) 



^ I,{T) + h{T) - -^UT) 



1 + 13 



1 + 13 



(3.7) 
(3.8) 



where 



h{T) 
h{T) 
h{T) 



T- 



1+0 



[x)dsvT{,dx) 



Tt-s{<^tXt{T - s)vt{-, s)){x)dsuT{dx) 



/ / Tt-s ( / Ts-u^tXt{T - u)du\ -v}^^{-,s) 

JK^ Jo \J0 / 



In the proofs of Theorems 2.1, 2.5 and in Theorem 2.6 for /5 < 1 we show 

V 



lim exp 

T^oo 



and 



l + P 



lim hiT) = 0, 

T^oo 



(3.9) 
(3.10) 

'x)dsh'T(ydx). (3.11) 

(3.12) 
(3.13) 



where f l3.13p is obtained from 

hiT) <^ f [ [ Ts{(pTu(p){x)dudsuTidx) 

i'x JM.'^ JO Jo 

(see (13. 5p ). In Theorem 2.6 for (3 = 1 the limit of l2{T) is non-trivial and corresponds to the 
expressions involving (see (I225D, fl^^ - fl^T^ . fl^^ ). In all the cases 



(3.14) 



lim h{T) = 

1 -^OO 



(3.15) 
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By the argument in |BGT6j . in order to prove 03.151) we show 



and 
where 



hm JUT) = 



hm J2(T) = 0, 



(3.16) 
(3.17) 



T-s 



1+/3 



< 



T 



< 



^2+2/3 , „ , 

rp JR"^ Jo 

WJ^ Jo 

1 



Ts-u \ Vt J Tu-r^rdrj du 
Tu i (p Tr-ipdr I du 



T 



\ 1+/3 

Tu-r^rdr j du 



x)dsuT{dx) 
x)dsh'T{dx), 

{x)dsh'T{dx) 



(3.18) 



7^ I / Trifdr 1 



v^o 



{x)dsh'T{dx) .{3.19) 



We remark that the proof of (13.151) is the only place where the high density (with specific 
conditions on Ht) is required in some cases. 

Finally, the =^ f convergence is obtained by an analogous argument as explained in the proof 
of Theorem 2.1 in |BGT4j . 

3.2 Auxiliary estimates 

Recall that the transition density pt of the standard a-stable process has the self-similarity 
property 

p^^(x) = a-'^/>t(a~i/"a;), xeR"^, a > 0, 



and it satisfies 



1 + \x\'^+°' 

where the lower bound holds for a < 2. 
Denote 



< pi{x) < 



l + \x 



d+a '■ 



fix) = / ps{x)ds. 
Jo 



The following estimate can be easily deduced from (I3.20p and (13.211) : 

C 



fix) < 



\x 



d+a '■ 



C if d < a, 
fix) < { C(l Vlog|a;|-^) if d = a, 
C7/|x|'^-" if d>a. 



(3.20) 

(3.21) 

(3.22) 

(3.23) 
(3.24) 
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We will also use the following elementary estimates: Let < a,b < d. If a + b > d, then 



/ 1 rr^d^ < VTTT-,- (3-25) 

jr"^ f y\ 



fI 12/ 



\i a + h = d, then 



If a + 6 < then 



/" I \-Y,dx<C{iy\og\y\-^) 

J\x\<i Ix-ym" 



(3.26) 



ixi<i 1^ z/hi^^i'' 



^ dx<C. (3.27) 



Now, let a > (i, < 6 < (i, then 

/ ^ rr^dx < --r. 3.28 

For d > 'J, denote 

f,iy)= [ fiy-x)\xrdx, (3.29) 



where / is defined in (13.221) . From the estimates above we obtain 

sup \y\''fjiy) < oo, (3.30) 
ls/l>i 

and 

{C if 7 < a, 
C(lVlog|y|-i) if 7 = a, (3.31) 
C/lyl^-" if 7>a. 

3.3 Proof of Theorem 2.1(a) 

According to the scheme sketched above, in order to prove =^f,i convergence we show (13. ip . 
By (l3.7l) -( l3TTTl) and (12.11) it is enough to prove (I3.12p . which amounts to 

/ .1 dx ^ ^ 



lim /i(T) = / / / Psix-y)[ Pu^siy)xiu)du ] dyds— [ ^{z)dz\ 

(3.32) 

(see (13.90 ) and, additionally, (I3.13P and (I3.15p . To simplify the notation we will carry out the 
proof for /i^ of the form fi^{dx) = \x\~'^dx instead of (II. 8p . It will be clear that in the present 
case {d < 7) this will not affect the result. 

By (13.90 . (13.20 . (13.60 . the definition of %, substituting s' = 1 — s/T,u' = 1 — u/T, we have 



hiT) 



PTs{x-y)[ / / PT{u-s){y - z)ip{z)x{u)dzdu] \x\ ^dydsdx. 
rrp ^ jRd Jo Jr"^ \Js Jm.'^ J 

(3.33) 
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Denote 

^y(x) = T'^/"y?(Ti/"x) (3.34) 
and ^ 

9s{x) = / Pu^s{x)x{u)du, < s < 1. (3.35) 

J s 

Observe that 

9s < Cf, (3.36) 

where / is defined by (13.221) . By (13.201) and (12. 5p . making obvious spatial substitutions in 
(I3.33p . we obtain 

h{T)=f f f p,{x-y){g,*^T{y)f^^\x\~Uxdsdy. (3.37) 
Jm.<^ Jo Jw^ 

Note that if we consider the measure fi-y of the form (11.81) . then in (13.371) instead of \x\~'^ 
we have (T"'*'/" + |x|''')~^. Since gs G L^{W^), by (I3.34p it is clear that gs * ^xiv) converges 
to gsiy) j^d^{z)dz almost everywhere in y. Hence, to prove (I3.32p it remains to justify the 
passage to the limit under the integrals in (I3.37p . Denote 

hT{y)= I [ Ps{x-y){gs*My)Y^^\xr'dxds. (3.38) 



First we prove pointwise convergence of Ht, which amounts to showing that the integrand is 
majorized by an integrable function independent of T. Fix y 0. We use (13.361) and observe 
that 





\x\Yif{T'^/"x) 




\x\ 


\d 



f*(pT{y) = / f{y - z)(pTiz)dz+ / fiy-x) — dx 

J\-A<\y\n J\-A>\y\/-i 

< f(l) [ M^)dz + f-, [ f{y-x)dz 

^ ^(Dl^w-'^ + FP- 

by the unimodal property of the a-stable density and since G 5(M'^). We conclude by noting 
that 



/ / Ps{x — y)\x\ "'dsdx <oo for ?/ 7^ 0, 
Jw.'^ Jo 



by (13:3T|) . 

Since (see (Km ) 

hT{y)< f,{ym*VT){y)f^^ (3.40) 

to prove convergence of hiT) it suffices to show that the right-hand side of (I3.40p (denoted by 
h*rp) converges in L^(M'^) as T ^ 00. 

If 7 < a, then is bounded by (13.311) . and the assumption (12.41) implies that 

/ G L^+\W^), (3.41) 
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f,m{\y\<i} e L^iM'^) (3.42) 



so (/ * fTy~^^ converges in L-^{M.'^). 

Next assume 7 > «. It is easily seen that ^rl{|y|>i} converges in L-^(M'^), by fl3.41l) and 
fl3.30p . To prove that h^{y)l^\y\^ij converges in L^(R'^) too, it suffices to find p,q > 1, 1/p + 
1/q = 1, such that 

and 

e L^iW^). (3.43) 

If 7 = a, then (13.311) implies that fl3.42p holds for any p > 1, and by (12. 4p it is clear that 
(I3.43P is satisfied for q sufficiently close to 1. 
If 7 > a, condition (12.41) is equivalent to 

7-« I il + /3)id-a) ^ ^ 
d d 

so we can take p and q such that 1/p + 1/q = 1, 

1 -f-a 1 (l + P)(d-a) 

- > — - — and - > ^ -. 

p d q d 

For such p and q we have (Km and ([333]) by ^M), (13:231) and (Km . 
This completes the proof of (I3.32p . 

We proceed to the proof of (I3.13p . By (I3.14p . applying the same substitutions as for /i(T) 
and using the notation (13.340 we have 



^ (-/'- 

Assume 7 < a. By (13.311) 



hiT) < C / f,iy)My){f * ^T)iy)dy. (3.44) 



TT rp2—d/a—'f/a 

hiT) < C,^—r^ pT(/*^T)lh 



< C'l 

by (12.41) . and since 



< Ci ^ IIV^I|l||/l|l+/3||<^T||i+£ ^ 



||^t||p= ||<^||pT('^/")(P-i)/P for p>l. (3.45) 
Next, let 7 > a. By ( K^ and fl3:30|l we have 

/2(T)<C2(/^(T) + /;(T)), (3.46) 



where 



and 



LT rp2—d/a—'y/a 

l2iT) = ^—, ||^t(/*^t)||i, (3.47) 

/;'(T) = ^tT^ J" /• , ^T){y)dy. (3.48) 
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By the Holder inequality, 

r,{T) < -^—-^ — ii^tIIpII/imiv^iIi (3.49) 

for any p, g > 1, 1/p + 1/g = 1. li 1/ q > {d — a) / d, then \\f\\q < oo, and if 1/q is sufficiently 
close to {d — a)/d, then by fl3.45p . fl2.5p and (12 ■4p the right-hand side of (13.491) converges to 
as T ^ oo. 

We estimate I2 (T) using the generalized Holder inequality 

h{T) < ^ ll/7l{M<l}llr||¥'T||p||/IUl¥'||l 

for r,p,q > l,l/p+ l/r + 1/q = 1. We take r,q such that 1/r > {'j — a)/d (then the r-norm 
will be finite by (I3.3ip ) and 1/q > {d — a)/d. By (12. 4p . it is easily seen that one can choose 
r, q as above and p = a /{2a — 7) + £ for e > sufficiently small (note that by (12. 4p . 2a; > 7). 
Then by (13.450 and (12.50 we obtain that (T) ^ as T ^ 00. Thus, we have proved (13.130 . 

According to the general scheme, in order to obtain (I3.15P it suffices to show (13.160 and 
(I3.17p . The proofs are quite similar to the argument presented above, therefore we omit the 
proof of (13.160 and we give an outline of the proof of (13.170 . since this is the only place where 
the condition (12. 6p is needed. 

By (I3.19p . (12. 5p and the usual substitutions we have 

7-l-{d/a)/3+{7/a)/3 

MT) < C R{T), (3.50) 

llrp 

where 

R{T)= [ f,{y)U*U*VTY^^f^^y)dy. (3.51) 
By (12.60 . to prove (I3.17p it remains to show that 

sup/2(T)<oo. (3.52) 

T 

If 7 < a then, by fIXMD . 

R{T)<C,\\fHf*^TrXX',<Cm\\X^^^^^^^^ (3.53) 

by dsm). 

If 7 > a, then we write 

R{T) 

'l?/l>i J\y\<^ 

By (I3.3O0 the first integral can be estimated as in (13.530 . and the second one is bounded by 

||j7-'L{M<l}||p||/||g(i+/3)||/|li+/3 IIV^IIl , 

where 1/p + 1/g = 1. We already know that there exist such p and q that this expression is 
finite (see fl3:A and ([333])). 
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We have thus estabhshed the convergence 

In order to obtain =^c convergence it suffices to show that the family {{X^, <f)}T>i is tight 
in C([0, 1],M) for any (p G ( [Mi] ) . One may additionally assume that > 0. We apply 

the method presented in |BGT3j and |BGT6] . We start with the inequality 

P{\{XT,<p®ij)\>S)<C6 {l-Re{Eexp{-i9{XT,'P®'^)}))d9, (3.54) 



valid for any ip G 5(]R),(5 > 0. Fix < ti < ^2 < 1 and take ip approximating — ^ti such 
that = ^jj{s)ds satisfies 

0<X<Mt,M- (3-55) 
Then the left hand side of (3.54) approximates 

Pi\{XTih),ip)-{XTih),^)\>6). 

So, in order to show tightness one should prove that the right hand side of (3.54) is estimated 
by 

C{t2 - tj)^+" for some h,a>0. 
By the argument in |BGT6] this reduces to showing that 

A(r) < C(t^-t?)i+" (3.56) 

and 

/i(r) <C(4-t^)^+", (3.57) 

where Ji is defined by f l3.9p . and 

AiT) = ^j^J^ ^ TT^si^%.^^)ix)x (l - ^) X (l - I) Ixrdudsdx. (3.58) 

The proofs of fl3.56p and (13.571) are quite involved and lengthy, therefore, as an example we 
present only the argument for the case 7 < a, which, together with (12.41) implies 



d < a^-^. (3.59) 

We start with (I3.57p . By self-similarity of ps we have 

/ psix -y)\x\-^dx <Cs-^/'^, yeW^. (3.60) 

Using this, (I3.37p . (3.35) and (I3.55P we obtain 

h{T)<C{W,{T) + W2{T)), (3.61) 
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where 



WiiT) = [ p^^siy - z)Mz)dzdu] dsdy, (3.62) 

Jr'' Jo \Jti Jr"^ J 

W^iT) = [ s-^/" f / V P-^^y - z)Mz)dzdu] dsdy. (3.63) 

JR'* Jti \Js Jw^ J 



max<' - - i J> < p < 1 (3.64) 



Fix any p such that 

max < 

a (5 

(see f l3.59p . For any fixed s G [0,ti] we apply the Jensen inequahty to the measure 



jll{r-s)-Pdr 
obtaining 

'tl 



W,{T) < [ r s-^/" ( f\r - s)-fdX f\u - s)-P{{u - sYp^.s * ^rm'^^dudsdy. 
Jr'' Jo \Jti / Jti 



We have 

\\Pu-s*^t\\i-,P < = («-.)-('^/")^/(^+^)|bl||l+/3||¥^||l, (3.65) 



hence 

■•i2 nil 



Jti Jo 

which, after the substitution s' = s/u, by fl3.64p and 7 < a, yields 

Wi{T) < Ci{t2 - hf^-p'^^it^^ - t\) < ^2(4 - ti)'+^'"''^^, (3.66) 

where h = 2 — j/a + pP — {d/a)(3 > by assumptions. 
Next, by fl3.63p we have 

r / / rt2-ti \ \ 

W2{T)<J^J^ ^"^^ VVi Puduj*My)j dsdy. 

The Young inequahty, substitution u' = u/{t2 — ti), and self-similarity imply 

W2{T) < c(4-"/"-ti-^/")(t2-ti)^+'^-('^/")'^ii/iii:?ii^iir 

< c^{t\-^''' -t\-^'y+^-^'"°'^^, (3.67) 

by dsn]). 

Combining ([MS]), (IMTD . (ITO]) and using (I339D . we obtain (l337j) . 
It remains to prove (13. 56^ . 
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Applying the usual substitutions to A{T) given by fl3.58p we obtain 
A{T) 

- y)ipT{y)'Pu-s{y - z)ipT{z)x{s)x{u)dxdzdyduds, 



F2 

JO Js 

hence, by f l3.60p . (13.551) and the Holder inequality, 

A{T)<C— -2 / / S~'^''^\\^T\\l+i.\\Pu-s*^T\\l+l3duds. 

Jti Js ^ 

We use (13351) . fl3:i5|) and ([23]), obtaining 

A{T) < - tl"'/")(t2 - 

which implies (13.561) by (I3.59p . This completes the proof of tightness. □ 

3.4 Proof of Theorem 2.1(b) 

We prove the theorem for 

j = d< —^a (3.68) 

(see Remark 2.2(c)). Recall that in this case k{T) occurring in (12. 9p and (I2.10p is logT. 

According to the discussion in Section 3.1 it suffices to prove (13.120 . (13.130 and (13.150 . By 
the form of the limit process (see (12.30 ). (I3.12p is equivalent to 

hm h{T) = / !\s{y){f\u-s{y)x{u)du)'^^dsdy ( [ ^{z)dz) ^\ (3.69) 

where a{Sd~i) is the measure of the unit sphere in R'^(= 2 if (i = 1). 

By (13.90 . (13.20 . (13.60 . using similar substitutions as in the previous section, we obtain 



h{T) 



Ps{xT /"-?/) / / Pu~~s{y - z)x{u)'^T{z)dzdu] dydsdx 



d 

(3.70) 



where is given by (I3.34p . We write 

Ji(T)=/UT) + /;'(T)+/;"(T), (3.71) 



where 



1 



IliT) = 7^1 [ I ■ (3.73) 



^OgT J\r,\>T^/<y Jo 

— f / 



I'liT) = r^l [ I .... (3.74) 
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Passing to polar coordinates in the integral with respect to x we have 

I[{T) = r [ f ! PsiwrT-'/" - y){g, * lp^{y)y+P^:^dydsa{dw)dr, 

logT Js^_^ Jo V ^ + r'^ 

where g is defined by f l3.35l) . The crucial step is the substitution 

' ^^^^ a 7c;^ 

which gives 

I[{T)= / / p,{wT^-y--y){g,*^T{y)f^^--—dydsa{dw)dr. 

It is now clear that if one could pass to the limit under the integrals as T — ^> oo, then /((T) 
would converge to the right hand side of (13.691) . This procedure is indeed justified by the fact 
that for / defined by (13.221) we have 

/ G L2+^(M''), (3.76) 

which follows from (13.681) . (I3.23P and (13.241) . We omit the details, which are similar to the 
argument in |BGT6j (see (I3.5ip therein). 

Next we show that Ii (T) and Ii (T) tend to zero. In (T) (see (I3.73P ) we substitute 
x' = xT~^/" and we use (I3.36p . obtaining 



^ ^1 11^ ~ l|l+/3 ^ C*i 11 11 1+^1 1 



0. 



I'l (T) (see (13.741) ) is estimated as follows: 

/;"(T) < [ f * if * ^rr^xT-'/^dx 



~ k^"'^"2+/3||(/ * ^t)^^^||(2+/3)/(1+/3) 

< i^ii/ii^siMir^-o. 



by (13776]) . This and fl3TT|) prove flMD . 

To prove (13.130 we use (I3.14p and easily obtain 

IJ '~r2—2d/a p 1 

hiT) < ^-^-^ / / * (Mf * ^T)){xT-yn-r-rrdd^- 

rrp J^2 i + \X\ 

We write the right-hand side as the sum of integrals over < T^^"} and > T^/"}. To 
estimate the integral over {|x| < T^/"} we use 

If 1 
sup / j — -:dx < 00, (3.77) 

T>2 logT J|^.|<Ti/- 1 + FI'* 
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and in the second integral we apply 1/(1 + {x]'^) < T '^Z". For each of the integrals we use 
appropriately the Holder inequality, properties of the convolution and fl3.45p . obtaining the es- 
timates C'iT2({'^/")i/(2+/3)-i/{i+/3)) and C'2T('^/")V{2+/3)-2/{i+/3)^ respectively (the factors involving 

negative powers of Ht and logT have been estimated by constants). These bounds tend to 
zero as T ^ oo by (13.681) . We omit details. This proves (IS.lSp . 

To prove (I3.16P and (13.170 we use (13.180 and (13.190 . Again, we consider separately the 
integrals over {|a;| < T^/"} and > T^/"}, and apply the same tricks as for hiT). 

For Ji(T) we obtain the estimate 

Ji(T) < ct('^/°)(i+/3)/(2+/3)-i _^ 

(logT and Ht appear with negative powers only), whereas 

T T 
J2iT) < C,—j——— + C2 



by assumption (I2.10p . The proof of Theorem 2.1 is complete □ 
3.5 Proof of Proposition 2.3 

Properties (a)-(c) are clear, following from (12.11) and Theorem 2.1(a). Recall that the index 
of self-similarity is defined as a G M such that the process (^ct)teK+ has the same distribution 
as {c°-^t)t€M.+ for any c> 0. 

To calculate the dependence exponent of ^ (see (12. lip . (I2.12p ) first note that by (12.11) and 
Proposition 3.4.2 of |ST] the finite-dimensional distributions of ^ are given by 



Eexp{i{zi^t, H h Zk^tk)} 

/ /. \ i/(i+/3) rt, i+fs 



^^Pl / E^M / Pri^ -y)\y\~^dy] i[o,t,]{r) pu-r{x)du 

X 1^1 - i sgn|^^2;j yj ^Prix - y)\y\~^dyj 



ft 



X l[o,t^](r)^%„_,(x)dM^ tan^(l + /3)^ drdx^ . (3.78) 

The argument goes along the lines of the proof of Theorem 2.7 of |BGT3] . For fixed 2; > 

and 0<'u<f<s<twe define = Dt{1, z;u,v, s,t) and = Dt{1, —z;u,v, s,t) (the 
formulas for D^, are obtained from (12.120 and (13.780 ). and we prove 



(jj-d/a if ^^^Yiei a = 2 or P> {d--f)/{d + a), 

(jj,-{d/a)6 for any (3<6<l+(3 + {d- -f)/{d + a) if a < 2, (3 < {d - -f)/{d + a). 



and for T sufficiently large, 

D+ > CT-('^/")'^ for any 5 > 1 (d-7)/(d + a)if a < 2,/3 < (d-7)/(rf + a). (3.79) 
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The upper estimates are obtained similarly as (4.3), (4.4) in |BGT3j and (13.1081) in [BGT6j . 
The only difference is that in formulas (4.9) and (4.10) in [BGT3] a new factor, J^dPr{x — 
y)\y\~"'dy, appears (which corresponds to Pr{x) in fl3.10ip in [BGT6|). This factor is responsible 
for the new long-range dependence threshold and the form of the dependence exponent fl2.13p . 
In the estimates we use fl3.30p . 

The first of the lower estimates is obtained exactly as (4.18) in |BGT3j . The new expression. 



/ / / Pr{x - y)\y\~^dydxdr, 

Ju J\x\<lJW^ 



that appears at the right-hand side is finite by fl3.3ip . 

To derive (I3T9D we argue as in (4.22), (4.24) of |BGT3j and we apply estimates (4.21) 
(which holds for |x| < T^^") and (4.23) therein, obtaining 

B+>CT~Wo^){i+P)+em+o^) / / p,^{x-y)\y\-^dydxdr, (3.80) 

where e > is sufficiently small. 

For 1 < \x\ < T'^/(d+»)<^-'^ we have 



/ 



Pr{x-y)\y\ ^dy > C\x\ ^ / pr{x - y)dy 

\x~y\<l/2 



> Ci\x\-^ inf inf pr{z) > CaT-'^^'/'^^'^+^^'^+^T. 



Putting this into (IHISOD we obtain (I3T9D . □ 

3.6 Proof of Theorem 2.5 

Each of the cases requires a different proof, and none of them is straightforward. We will 
present a detailed proof of the part (a) only. In the remaining cases we will confine ourselves to 
explaining why the limit processes have the forms given in the theorem (recall that part (b) (iv) 
has been proved in |BGT6] ). It seems instructive to compare the proofs for this theorem to 
the argument given in the proof of Theorem 2.1(b) for j = d. Although the critical cases are 
of different kinds, some of the technical tricks repeat in all cases, nevertheless they are applied 
in a slightly different way and are far from being identical. 

Proof of case (a) To simplify calculations we again consider the measure fi^ of the form 
fi^{dx) = \x\~'^dx instead of (11. 8p . 

In (13. 9 p we substitute u' = s — u and then s' = {T — s)/T, obtaining 

hiT) = ^^^f f [ Psxix-y) I [ [ p^{y - z)(p{z)x(s + ^)dzdu\ \x\~^dydsdx. 

F,f^^ Jr^ Jo jRd \Jo Jr^ J 

Using f l2.15p . f l3.20p and substitution x' = x(sT)~^/" we have 
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where 



/ / ~ ^)v^('2)x( -5 + 77; I s "'dydsdx 

Jo ilRd \ TJ J 

i[{T) + i';{T) + i';'{n (3.81) 

im^^i n ... (3.82) 




"'l<|j/|<Tl/a 
1 



J('(T) = -i- / / / ... (3.83) 

log T ./rod ./o i|i^|>Ti/« 



Passing to polar coordinates in the integral with respect to y and making substitution 03.751) 
we obtain 

I[{T) 

-1 i>l/a 




pi pl/a p 

I / / pi(x- wr"-i/"s"i/") 

Jo JSd^i 

rm-s) r. fu \ _ _ 

1 / VuiwT'^ — z)ip{z)x\ — + s\dzdu\ s "'T'^'^a{dw)drdsdx. 
Jo Jr'^ \T / 



We substitute z' = T-''z, u' = uT-""^, use (jSSQ]) and flCTj) . arriving at 

I[{T)= [ [ [ [ pi{x-ws-^'^T''"^/^){hT{r,s,w)f^^ s-^/''\x\-^a{dw)drdsdx, 
Jw^ Jo Jo Jsd-i 

(3.85) 

where 

„Ti-'-(l~s) p 

hT{r,s,w)= / / pu{w - z)T'^'^^{zT')x{s + uT'"-^)dzdu. (3.86) 

Jo Jr"^ 

It is clear that on the set of integration one should have 

lim hT{r,s,w) = / (p{z)dz / pu{w)dux{s) 

T^oo had n 



= Cd,a / v{z)dzx{s), (3.87) 
where Cd,a is given by fll.l3p . which should yield 

[{T) = Cll^-a{Sd-i) I pi{x)\x\-^dx f s-'<''^x'^\s)ds(! v{z)dz\^'' . (3.88) 

a had n \ had / 



lim / 
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However, fl3.87l) and fl3.88p need a justification. It is easy to see that the first integral in fl3.86l) 
can be replaced by du. Since 

lim / Puiw — z)T^'^ip{zT^)dz = Pu{w) / (p{z)dz, 

it is clear that in order to prove (13.871) it suffices to show that 

sup sup / Pu{w ~ z)T'^'^ip{zT^)dz 

is integrable in u. This is clear for m > 1 since d > a, and for m < 1 we argue similarly as 
in fl3.39l) obtaining an integrable bound CiPu{w /2) + €2- In the same way one shows that 
hxir.s.w) < C. This together with fl3.87p easily implies fl3.88p . 
Next, it is easy to see that for /(" defined by f l3.84p we have 

IiiT) < 7^ - 0. (3.89) 
logT 

A little more work is needed to prove that also 

hm I'liT) = 0. (3.90) 

1 — >oo 

By dsn, 

I'iiT) < f ill Pniy - z)ip{z)dzdu\ dy 



< C,{R,{T) + R2{T)) 



where 



Ri{T) = - — - / \ u Pu{y - z)ip{z)dzdu ] dy, 

i?2(T) = - — - / \ u Puiy - z)Lp{z)dzdu] dy. 

logT Jlyl>T^/. Wo J\z\>^ J 



We have 



logT 

after obvious substitutions and using (I2.14p . Hence lim -Ri(T) = by (I3.23p . Furthermore, 

T — ^00 

R2(T) < I ( r I puiy - z)^-^^-^dzdi}\ dy 

' - logT^i^^v. ^ J\A>^ J 



J^(2/a)(l+/3) ' 
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since, under f l2.14l) . 



sup-^— / ( / / Pu{y - z)ipi{z)dzdu] dy < oo, 



T logT J^d \Jq 

for any ipi e S{R'^) by (13331) in |B(;T4] (in our case ipi{z) = \z\'^ip{z)). This proves f l3.90p . and 
by (ESID-iESD, (I33HD and (131891) we have established (137[2|) . 

To prove (13.131) we use (13.141) . which, after standard substitutions, gives 

h{T) 

< C'i^-'/('+^)T-2/(l+/^)+27/«(l+/3)+2-d/a-7/a(l0g2^)-2/{l+/3) f f^^y)^^^y)^f^^^)^y)dy 

(see ( 127[5|) . (13:221) . (13129]) . (131341) ). By (133TD we have 

The assumptions (12.141) and 7 < a imply that 

2 27 7 1 

2 + - + - = 



1 + /3 a(l + /5) a a /3 

for some 6* > 1. Observe that 

/ G /.'^(R'^), 1 < g < 1 + /3, (3.92) 

by (13:231) . (13:21) and (ICTD . Fix g > 1 such that (1 + < g < 1 + /5 and p = q/{q - 1). By 
the Holder inequality and (I3.45P we obtain 

h{T) < Cir-^/^+«i+'3)/^)(i/^)||v;||p ^ 

as T — >• cx), by assumption on g. 

To prove (13.161) we use (13.181) which, by an analogous argument as for /2, gives 

Ji{T) < Cr-i+^/"-(i+'3)/^||/* (^t(/*^t))||J:?. (3.93) 

Taking g < 1 + /5 sufficiently close to 1 + /5 and using (I3.92p . the Young and Holder inequalities 
can be applied to conclude that, by (I3.45p . 

ii/*(^T(/*^T))iiis<o(n, 

where r < 1 — 7/a + (1 + /3)//3; we omit details. This and (I3.93P yield (13.160 . 
To prove (13.170 we write an estimate similar to (I3.50p . namely, 

J2{T) < Cr^(^/"-^)i?(r), (3.94) 

where R{T) is defined by ( 13.510 . In the present case (I3.52p does not hold, but similarly as 
before, using the Young inequality, (I3.92p and (13.450 it can be shown that -R(T) = O(T^) for 
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any e > 0. This and fl3.43p imply fl3.17p since 7 < a. The proof of part (a) of the theorem is 
complete. 

Sketch of the proof of case (b) 

(i) We repeat the argument as in (13.811) - fl3.86p with Ft given by (12.161) . Again, it can be 
shown that the only significant term is I[{T), i.e., lim-r^oo -^1 (7") = lim-r^oo -^1 (7") . In order to 
derive this limit, in (I3.85P we substitute s' = sT^'"^", obtaining 



l/a pT 



1 — ra 



I[{T) = - — -/ / / / pi{x-ws^'''){hT{r,sT-^^'''',w)Y+'^s-^\x\"'a{dw)dsdrdx. 

iog J- Jw^ Jo Jo Jsa-1 

It is easy to see that the hmit remains the same if the integral ... (is is replaced by 

j-^ ...ds. Let I'i{T) denote /((T) after this change. Next, we substitute s' = logs/ log T 
and we have 

I[{T)= / / / Piix-wT-'/'')\x\-''{hT{r,T'~^^^'',w)y+'^dxdsa{dw)dr. 

Jo Jsd-i Jo Jr"^ 

By (13.860 . it is clear that on the set of integration 

hm hT{r,T^-'^^'',w)=C,,^ [ ^{z)drx{0). 

This shows that we should have 
hm /i(T)= hm I[{T) 

1 — >OQ 1 — >OQ 

= Cll^o{Sa-i) j pi{x)\x\-''dx f'\l-ra)dr(l ^{z)dz\^\^+^{Q), (3.95) 
JR'^ Jo ^Jr'' ' 

and this passage to the limit can be indeed justified. The right-hand side of (13.950 is equal to 
log£^exp{—C(X, '?/')}, where X(= KXd) is the limit process defined in the theorem. We 
skip the remaining parts of the proof. 

(ii) In (13. 9p we substitute u' = s — u and then s' = T — s, obtaining 

/i(T) = /((T) + /('(T), 

where 

logTJi jRdJj^d 

Puiy - z)Lp{z)x(^^^^Y-)dzdu] dydxds 



and 



I';{T) = ^ [' [ j ... dydxds. 
log T Jo Jr-^Jr-^ 
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It can be shown that 



Um J('(T) = 0. 

1 — >oo 



In I[{T) we substitute x' = xs ^^^^ = ys -^Z^, = u/s^ and use (I3.2UI) . which gives 

x(^y J ^Puiy-zs '")(f{z)x\^ jdzduj dydxds. 

By ^rm . s-(rf/«)/3-7/"+i+/3 = s-i^ so, the substitution s' = logs/logT yields 

\ 1+/3 

- zT-'/'')(f{z)x (((m + l)r'^"^) dzdu ] dydxds. 



(3.96) 

It is now seen that one should have 
lim UT) = lim I[(T) 

= C'Jf [ [ p,ix-y)\xr\y\-^''--^^'+^^dxdy( [ ^(^)rfzV^'' x'-'^W (3.97) 



d Imd 



Note that the integrals are finite by fl3.2ip . (13.281) and (I2.18p . The justification of (I3.97P requires 
some work, but we omit it for brevity. 

(iii) As (i = 7, we must keep the measure /x^ in its original form (11.81) . 

Arguing as in the proof of (ii) and taking into account (I2.16p . instead of (I3.96P we obtain 

1 r r n^sd/a 

Ii(T) = -— / / / p^{x-y) 



logTio k^k^ ' ^^l + IxT^/-!"^ 

Pu{y - zT''/'')ip{z)x{{u + l)T'-^)dzdu ] dydxds. 





Since 

lim — — / piix — y) , ^ , ,, dx = s—G{Sd-i]Vi{v), 

it can be shown, with some effort, that 
lim UT) = lim /((T) 



Cl'f^cris,^,) [ p,iy)\y\~^'-'^^^'^^^dy( [ ^(^)rfz)'^V+^(0). 
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Again, we omit the remaining parts of the proof. 



3.7 Proof of Theorem 2.6 

We only give an outhne of the proof. The following lemma is constantly used. 
Lemma Let ip E S{R'^),Lp > 0. 

(a) If d > a{2 + + /?), then the functions G(f,G{G(py^^ and G{G(pY are hounded. 

(h) If d > a{l + (3)/(3, then additionally and (G'(G(^)i+^)i+'^ are mtegrable (and 

bounded). 

(c) If a < 'J < d and d > a{2 + (3) / (3 — / [3 , then additionally to the properties in (a), 

f G{Gvy-'^{x)-^—-dx<^. 
This Lemma follows easily from (11.121) and (I3.25l) - (l3.28p . 

Proof of part (a) of the theorem. As before we consider ii^{dx) = \x\~'^dx. In (13.91) we 

substitute u' = s — u, then s' = (T — s)/T and, finally, x' = xT^/"s~^/°', obtaining 

h{T) 

= [ [' [ Pi{x-ys''/''T-'/'')l '^TMy)x(s + ^)du] s-'^/'^lxrdydsdx 
Jr" Jo Jr" \Jo J 

(see (I2.24P ). It is easily seen that by part (b) of the Lemma we have 

lim Ji(T) = / pi{x)\x\-''dx [ s~^<'''x^^^{s)ds [ {G^{y)f+^dy. (3.98) 







For (3 < 1 this is exactly logE'expj— C(X, ® V")}) where X is the limit process described in 
the theorem. Moreover, in this case (13.141) and boundedness of Gip easily imply 

For /3 = 1 we use (I3.10p and (13. 4p . obtaining 

h{T) = i',{T)-n{T)-'^i2{n 



where 



/2(T) = ^[ f I ps{x -y)\x\ ^dxp{y)x{s) I Tu-s(p{y)x (^) dudyds, 
Jo Jr'' Jr'' Js ^ 



H' 



m = ^-^111 PT-six - yMy)xTiT - s) 
Jr'^ Jo Jr"^ 

X / Ts^u{'^x{T ~ 'u)'^T{--,u)){y)\x\~"'dudydsdx 
Jo 

J'-i'iT) = f f I Pt-s{x - y)p{y)xT{T - s) f T,_^{vl{-,u)){y)\x\~'^dudydsdx. 
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T 



Ft JR" Jo -'r" -'o 



Substituting u' = u — s,s' = s/T, and then x' = xT^/"s^/" and using part (a) of the Lemma 
and fl2.24p we have 

hm /^(T) = / pi{x)\x\~"'dx [ s'"'/''x\s)ds [ <^{y)G^{y)dy. (3.99) 



T^oo 

Applying fl3.5p . fl2.24l) and the Lemma above we get 

l2iT) < [ [ [ Ps{x - y)\x\~^^{y)G{^G^){y)dydsdx 

< C'lT"^^/^)^^"'^/") 0, 

and, analogously, 

I'^T) < ^[ f I Ps{x-y)\x\-^^{y)G{{G^Y){y)dydsdx 

< (72^^^^/^)^^"'^/") 0. 

This and fl3M . ^W\f imply that for = 1 the limit of {V/2)Ii{T) + hiT) is exactly 
\ogEexp{—G{X,ip^ilj)}. Similar estimations, together with the Lemma, yield fl3.16p and 
(I3.17p . This completes the proof of part (a) of the theorem. 

Proof of part (b) of the theorem. Following the general scheme one can show 



hm hiT) = / pi(x)|x|-"rfxx^+^(0) / {Gipy^^iy)dy, 

lim hiT) = cp pi{x)\x\~°'dxx^{0) / ip{y)Gip{y)dy, 

and (13.161) and fl3.17p (recall that C/j is defined by (12.260 ). This is accomplished by an argument 
similar to the one used in part (a). Due to the criticality (7 = 0), the integrals . ..ds in 

(l3.9p -( l3rTTi) require a different treatment. They are split into ...ds + ...ds; the first 
summand converges to zero, and in the second one we use the substitution s' = logs/ log T. 
Here, again, we use repeatedly the Lemma above together with the easily checked fact that 



1 



T 



sup- — — / / Tsh{x)\x\ ""dsdx < 00 
T>2 log T J^d Jq 

for any integrable and bounded function h (recall that d > a). We omit details. 

Proof of part (c) of the theorem. Recall that the case j > d has been proved in |BGT6j . 
For a < 7 < c? we use the Lemma (part (c) is particularly important). We show 

lim /i(T) = / GiGipY+f^ix)fi^{dx)x^+^{0), 

T^oo had 



hm h{T) = Cfs f G{^G^){x)fi,{dx)x\0), 

T^oo J^d 
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fl3.16p and fl3.17l) . Here /i^ is either given by fll.81) or, for 7 < (i one can take jj,^{dx) = \x\ '^dx. 
Again, the details are omitted. 



3.8 Proof of Theorem 2.8 



The proof is based on two general lemmas which, hopefully, are of interest by themselves. 

Lemma A. Let Y be an {d, a, P)-superprocess with Yq = fi and N be the empirical process of 
the corresponding branching particle system. If for any bounded Borel set A C M'^, 



then also 



/ Yt{A)dt < 00 
Jo 



Nt{A)dt < 00 



L^o 



1. 



(3.100) 



(3.101) 



Proof of Lemma A. Let C, denote the standard a-stable Levy process in R , and let be a 
Markov process with semigroup 



Stl^i^X) = Er, 



exp 



where if; is a fixed element of C^(R'^) (bounded support), ip > 0. The process C, takes values 
in U {t}, where f is a cemetery point where it remains after killing by exp{— ijj{(s)d.s}. 
The infinitesimal generator of C, is 

Aip{x) = (Aa - i){x))Lp{x). 

Let Y"^ be a superprocess in M*^ constructed from ^ and (1 + /5)-branching, with Y^ = ^. 
The Laplace functional of its occupation time is given by 

Eexp|-£(F/,¥,)ds| =exp{-(/i,«J(t))}, ^eC^iR"), ^ > 0, (3.102) 



where u'^{x,t) is the unique (mild) solution of 



d 



V 



Qul{x,t) = (A,-^(x))<(x,t)-^«(x,t))^+^ + ^(x) 



(3.103) 



<(x,0) 







(cf. (11. 6p ). The Laplace functional of the occupation time of the process is given by 

Eexp |- ^ {N,, = exp{-(/i, v^{t))}, ^ e C^(M^), ^ > 0, (3.104) 
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where v^p{x,t) is the unique solution of 



d 



V 



V 



(3.105) 



f<^(x, 0) = 



(cf. (HI!)). Equations fl3303D and fl3:T05D coincide for = V, hence, from fl3102l) . (13101) . 

£;exp|-^*(F/,^)cis| = Eexp|-£(iV„^)ds| =exp{-(/x,wJ(t))}. (3.106) 

The superpro cesses Y and Y^ are obtained as (high-density/short-hfe/small-particle) 
limits of the same process N ^ removing first the killed particles in the case of Y^ . Hence 



£^exp 



^)ds ) > Eexp { - / (r,^ (p)ds } for all t 



(3.107) 



for any ^9 > 0, in particular for ip = ip, hence, from fl3.106p . f l3.107p . and (13.1021) with ip = 0, 

exp{-(/i,Mj(t))} > exp{-(/i,M°(t))} foralH. (3.108) 
Taking fi = 6x in (I3.102p we see that 

u^{x,t) y u^{x) and m° (x, t) /" -u^ (a;) as t /" oo, 
hence, from (I3.108p . 



(3.109) 



From (13A00|) . (13710611 . (13A09D . 



P 



{Yt,'4')dt < oo 



limexp{-(/x,ML)} 



9\0 



{Nt,ip)dt < oo 



so (13.1011) is satisfied for any bounded set A C R'^ and the lemma is proved. 



□ 



Lemma B. Let N be the empirical process of the {d, a, f3) -branching particle system with locally 
finite initial intensity measure fx. If 113.101]) is satisfied for any bounded set A C M!^, th 



en 



P[sup{t : Nt{A) > 0} < oo] = 1 



(3.110) 



for any bounded set A. 



Proof of Lemma B. Let Bji be a closed ball in with radius R centered at the origin. Let 
(tj, Xi, Tj), z = 1, 2, . . . , be a sequence of random vectors defined as follows for any realization 
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of the branching particle system. First we exclude all the particles which start inside Bji 
at time and their progenies. Let ti be the first time any of the remaining particles enters 
Bji,xi is the entry point, and Ti is the occupation time of the closed ball Bi{xi) of radius 1 
centered at xi by the tree generated by the entered particle. We exclude this tree from further 
consideration. Let t2 be the first time after ti that any of the remaining particles enters B^, 
with X2 and T2 defined analogously as above; and so on. Let rj denote the total number of first 
entries {ti,Xi,Ti),i = 1,. . . ,1]. We will show that r] < 00 a.s.. Suppose to the contrary that 
P[r] = 00] > 0. By construction, Yll=i '^i — Io° ^t{BR+i)dt, hence ^^^^ < 00 a.s. by fl3.10ip . 
By the strong Markov property and homogeneity of the motion, conditioned on {77 = 00} the 
random variables are i.i.d.. Hence 




and this is a contradiction since, as observed above, P[J2ili = 00] = 0. 

Going back to the particles that start inside Br, there are only finitely many of them since 
Ii{Br) < 00. 

In conclusion, with probability 1 only finitely many initial particles generate trees that 
contribute to the occupation time of any given bounded set, and all those trees become extinct 
a.s. in finite time by criticality of the branching. So (13.1101) is proved. □ 



Now, to prove Theorem 2.8 it suffices to observe that under its assumptions the corre- 
sponding superprocess Y suffers local extinction by Theorem 3/3 of [12], hence fl3.100p is clearly 
satisfied and the theorem follows immediately from the lemmas. 



3.9 Proof of Proposition 2.9 



First observe that it suffices to prove convergence of finite-dimensional distributions. Indeed, 
in the proof of Theorem 2.1(a) we have shown tightness of X-r = — EZx, and the presence 
of high density was not relevant in that proof. On the other hand, from Proposition 2.1 of 
|BGT5] it follows easily that the family of deterministic processes {E{Zt,^))t>i is tight in 
C([0, r],R),r > 0. Hence tightness of Zt follows. 

Without loss of generality we assume that r = 1. Fix < ti < t2 < • • • < < 
1, yji, . . . , G iS(R'^), and we may additionally assume that (^i,...,v3„ > 0. In order to 
show convergence we prove that 

lim Eexp <( -y^(ZT(tfc),v?fc) > = exp <^ - / v{x,l)\x\'^ dx\ , (3.111) 
[ J J 

where v satisfies (2.36) with ip given by (2.37) for 6k = J^d (pk{y)dy (as explained in |Taj . the 
solution of (2.36) is unique.) 

For simplicity we consider fi^{dx) = \x\~'^dx (it will be clear that the limit is the same as 
for /x^ given by (1.8)). Also, to simplify the notation we take = ... = </?„ = y?. Essentially 
the same argument can be carried out in the general case. 

As in [Taj and |BGT4] (the possibility to pass from space-time random variable to the 
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present situation) we have 



(3.112) 



where v satisfies (3.4) with x(t) = X]fc=i l[o,tfc](^)) Xt(^) = x{'t/T). Formula (3.112) is an 
analogue of (3.7), and its form is simpler since now we do not subtract the mean. 
The right-hand side of fl3.112p can be written as 



exp 



hT(x,l)\x\ '^dx 



where 



hT{x,t) = T'^/"-^/"wT(a;T^/",Tt), < t < 1. (3.113) 

To prove (3.111) it suffices show that /it(", 1) converges to f (■, 1) in L-'^(]R'^, \x\~'^dx); in fact, we 
will prove that 

hr^v in C{[0,l],L\R'^,\x\'^'dx)). (3.114) 
By dSmSD, dSaD, fl^:^ and we have 



hT{x,t) 



%^,T''^^{T"'^-){x)x{l-s)ds 



T^/^ / %^s{y^iT'/^-)hT{;smx)x{l - s)ds 
Jo 

V 



1 + /3 



%_s{hT{-,s)f+\x)ds. 



In particular, this implies that 



hT{x,t) < CT'^/'' [ %^{T^/^-)ds. 
Jo 



(3.115) 



(3.116) 



Let 



RTix) 



I %^s{T'"''^{T^'''-)){x)x{l-s)ds - f pt-s{x)x{l-s)ds I v{y)dy 
Jo Jo 

(compare with the first formula on page 851 of [Taj ) . We will show that 



sup 

t<i 



lim / Rt(x)\x\ '^dx = 0. 



(3.117) 



(3.118) 



Applying the usual substitutions and the fact that Pu{x) is a decreasing function of we 
obtain 

»i 



RT{x)\x\-^dx <C I I y^{y) 



{p.,{x-T-^/^)-p.,{x))d 



u 



\x\ "'dydx 



< Cj v{y) I \f{x-T-^'-y)-f{x)\dxdy 

"(/ 

36 



+ C( / / ^{yM-^Mydx 

\x\<l 1^'^ 



^{y)\fix-T-'/"y)-f{x)\^dydx 



1/9 



where / is defined by fl3.22l) . and p,q > 1 are such that 7p < d,{d — a)q < d, 
1/p + 1/q = 1 (such p and q exist since 7 < a < rf). Hence ( 13.1181) easily follows from fl3.2; 
and (1^ . 

Next, we will show that 



/ sup 






Jo 



\x\ '^dx = 0. 



(13.1161) and (13.311) imply that the expression under the lim can be estimated by 

T^/"/ f,iyMT'/'^y){f * ^T){y)dy < T^/"||¥.(tV'^-)IIimII/IIi+/5||¥'IIi 

(we have used (l233|l and / G L^+^ by (13:231) and (13121) ). 
To prove (3.114) we check the Cauchy condition, i.e., 

J(Ti,T2):= / ?,UY>\hT^{x,t) — hT2{x.,t)\\x\~'^dx ^ Q as Ti,T2 ^ 00. 

jR'i t<l 

Using (3.115), (3.118) and (3.119) we have 



(3.119) 



(3.120) 



V 



1 + 13 



(3.121) 



where lim Ji(Ti,T2) = and 



J2(Ti,T2)= / sup / Tt^s\h^'^{-,s) - h^'^{-,s)\{x)ds\x\ ^dx. 

JR'I t<l Jo 

By (13.311) and the Holder inequality, 

J2in,T,) <C [ sup \h'+^{y,t) - h'+^iy,t)\dy 



sup \hT,{-,t) - hT^{-,t)\ 



1+/3 



sup |/iTi(-,i)| 



t<l 



sup \hT2{-,t)\ 



Using (13.1161) and the fact that / G it is easily seen that 

sup hT{-,t) 

t<i 

To show (13.1201) it suffices to prove that 



sup 



< 00. 



lim 



sup \hT,i-,t) - hT,i-,t)\ 

t<l 



(3.122) 
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This can be derived in a similar way as in |Taj (see (I2.2ip and subsequent estimates therein). 
The only difference is that the term corresponding to hiT) in |Taj requires a slightly more 
delicate treatment; in our case it has the form 



j(7/«)(l+/3) 



sup 



Z_^{^{T'/^-)hT,{-,u)){x)du 



1+/3 



dx. 



Using (13.1161) and the Holder inequality this is estimated by 

where q = d/ {d — a + e),p = d/{a — e), and e > is such that 7 < a — £. Then the right-hand 
side is not bigger than CT^^^^^^/°'^^'^^'^~°'\ which tends to zero as T — > 00. 

Combining (13.1201) . (I3.118p . (13.1191) and (I3.122p . it is seen that one can pass to the limit in 
(13.1151) letting T — > 00, thus obtaining that the limit of satisfies (12.360 . This proves (13.1140 
and completes the proof of the Proposition. □ 



Proof of Theorem 2.11 



The proof is similar to those for the particle system, starting from an equation analogous 
to (3.7)-(3.8), where Xt is now defined for the occupation time fluctuation process (1.10) 
corresponding to the {d,a,j3,'j) superprocess Y, and equation (13.40 is replaced by 

VT{x,t) = / Tt^. 

Jo 

where the term hiT) in equation (13. Sp . given by (I3.10p . does not appear. This reflects the 
fact that comparing the log-Laplace equations (1.4) for the particle system and (1.6) for the 
superprocess, the term —ipv^p is missing in (1.6). (Equation (3.123) can be obtained from 
(3.4) by the same limiting procedure that yields the superprocess from the branching particle 
system. An equation analogous to (3.7) for the superprocess can be derived from continuous 
dependence of the occupation time process with respect to the superprocess, and continuity of 
the mapping C([0, r], 5'(M'^) 3 x^x e 5'(R'^+^) in (1.11) (BGRj .) It follows that the results 
for the superprocess are the same as those for the particle system, except in the cases where 
hiT) has a non-zero limit, and to obtain the results in those cases it suffices to delete those 
non-zero limits. Therefore the limits in Theorems 2.1 and 2.5 are the same for the superprocess, 
and for those in Theorem 2.6, C/3 = in all cases. □ 



(PtXt{T - u) 



V 



1 + /3 



,u) 



{x)du, 



(3.123) 
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